1. Technical Field
The present invention relates in general to a system and method for heuristically optimizing a large set of automated test sets. More particularly, the present invention relates to a system and method for heuristically adjusting input parameters based upon output variables in order to meet one or more performance goals.
2. Description of the Related Art
An automated test system usually includes two primary components which are a system under test and a system automation engine. A set of input parameters and scripts drives the system automation engine which, in turn, drives the system under test. A typical system automation engine establishes environmental conditions, such as particular load environments, and tests the system under test using a set of input parameters. While testing, the system automation engine collects a set of data (i.e. output variables) from the system under test. After testing, the system automation engine may analyze the data to identify trends or parameters that exceed preset limits.
A user may wish to implement performance testing in order to identify how to optimize his computer system when the computer system is executing a particular application. During performance testing, a system automation engine uses a set of software modules with a large number of test sequences to characterize a system under test. Each test sequence has a unique set of input parameters and a specified data load. Given the complexity of many computer systems, a challenge found is that a numerous amount of test sequence executions are required in order to adequately characterize a system under test. In addition, another challenge found is in analyzing and interpreting the vast amount of resulting test data. Quite frequently, it becomes unfeasible to execute a complete sequence of tests. Therefore, a system automation engine typically uses a subset of input parameters and data loads to test a system under test that represents boundary conditions. However, a challenge found is that this approach may not capture a computer system's optimum performance.
What is needed, therefore, is a system and method to automatically identify input parameter values that results in a system under test's optimum performance.